Complexity results for factoring univariate polynomials over the rationals (version 0.3)

In [6] Zassenhaus gave an algorithm for factoring polynomials f ∈ Q[x]. In this algorithm one has to solve a combinatorial problem of size r, where r is the number of local factors of f at some suitably chosen prime p. This combinatorial problem consists of selecting the right subsets of the set of local factors. In the worst case, the algorithm [6] ends up trying 2r−1 such subsets (if a subset has been tried then one can skip its complement) so this algorithm has an exponential worst case complexity. Let N be the degree of the polynomial f . Most of the time the number r is much smaller than N , which explains why Zassenhaus’ algorithm is often fast despite its exponential worst case complexity. To observe this exponential complexity on a computer, take polynomials of high degree for which the Galois group contains only elements of low order (worst case are the Swinnerton-Dyer polynomials, whose Galois groups have only elements of order 1 and 2, consequently, these polynomials have r as high as N/2). In [4] Lenstra, Lenstra and Lovász introduced a lattice reduction algorithm, which we shall refer to as the LLL algorithm. The paper [4] also gave a factoring algorithm which avoids the above mentioned combinatorial problem by constructing factors of f using LLL. The result was the first polynomial time algorithm for factoring polynomials in Q[x]. Schönhage [5] gave a sharper complexity result for a similar approach. The paper [3] gave a factoring algorithm that uses LLL as well. What was new is that LLL was not used to construct the factors (constructing factors is a problem whose size depends both on the degree as well as on the size of the coefficients). Instead, LLL was only used to solve the combinatorial problem (a problem whose size depends only on r, because subsets of a set with